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Abstract 

This paper introduces Ingen, a polyphonic modular host 
for LV2 plugins that is itself an LV2 plugin. Ingen 
is a client/server system with strict separation between 
client(s) and the audio engine. This allows for many dif¬ 
ferent configurations, such as a monolithic IACK appli¬ 
cation, a plugin in another host, or a remote-controlled 
network service. Unlike systems which compile or ex¬ 
port plugins, Ingen itself runs in other hosts with all edit¬ 
ing facilities available. This allows users to place a dy¬ 
namic patching environment anywhere a host supports 
LV2 plugins. Graphs are natively saved in LV2 format, 
so users can develop and share plugins with others, with¬ 
out any programming skills. 
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1 Introduction 

The Free Software world has long had pow¬ 
erful visual programming environments like 
Pure Data [Puckette, 1996], and higher level 
modular synthesizers like Alsa Modular 
Synth [Nagorni, 2003], However, most exist¬ 
ing software modular environments (or simply 
modulars ) do not integrate as well as possible 
with other projects. Due to the limitations of 
popular plugin APIs, most existing modulars are 
primarily designed around built-in internals, and 
use a different interface themselves (e.g. running 
only as an application). This situation results in 
much effort spent building components that are not 
widely useful across applications. 

There are several typical forms for audio proces¬ 
sors: a stand-alone software application, a software 
plugin, or a remote device. Each has advantages de¬ 
pending on the situation. Remote control is neces¬ 
sary for hardware to integrate with a software envi¬ 
ronment, and increasingly popular for software due 
to the pervasiveness of tablets, powerful controllers, 
and networks. An ideal system must be controllable 
from any location to fit well in all these scenarios. 
Consequently, the same must be true of the plugins 
hosted within it. 


This leads to the elegant conclusion that the ideal 
form of a modular plugin host, and the ideal form of 
a plugin within it, are one and the same. Ingen is an 
exercise in chasing this ideal: a modular host that 
has exactly the same external form as the plugins 
used within it. The practical benefit of such a design 
is that the user can build a device anywhere in the 
system where plugins are supported. This makes it 
possible to work around limitations in programs or 
the lack of an available plugin to solve the necessary 
problem. By making it simple for users to share 
their creations, the community at large can benefit 
from the pool of plugins created by users who would 
not have done so if writing code was required. 

Ingen takes advantage of the LV2 plugin API's 
extensibility to achieve these goals. The two have 
a symbiotic relationship: when Ingen itself requires 
an API advancement, the improvement ideally be¬ 
comes standardised in LV2. Other plugins may then 
use this API, resulting in more powerful plugins for 
use in Ingen, or other hosts. Likewise, Ingen bene¬ 
fits from advancements originally designed for other 
plugins. 

This paper introduces Ingen as a useful tool for 
users, and shares the general conclusions reached 
over the years that led to its design, and conse¬ 
quently the design of many aspects of LV2. 

2 Features and Philosophy 

2.1 Internals Considered Harmful 

Ingen is designed around the principal that generic 
plugins should be used wherever possible: internals 
are a symptom of an inadequate plugin API. With 
an open and extensible specification like LV2, these 
limitations can be addressed so progress needn’t be 
stalled. This avoids a walled garden effect where a 
large amount of effort is spent on internals that only 
work in one program. 

The minimalist ideal is for Ingen to have no 
internals whatsoever, but currently a few are re¬ 
quired for tasks that are beyond the capabilities of 
generic plugins. In particular, LV2 currently lacks 


polyphonic voice control, so the Note internal per¬ 
forms voice allocation, sending controls to partic¬ 
ular voices based on MIDI input. Perhaps in the 
future there will be sufficient developer interest in 
(visibly) polyphonic plugins to develop an LV2 ex¬ 
tension which will eliminate the need for a special 
voice allocation internal. 

2.2 Messages Considered Wonderful 

The common combination of GUI-centric design 
and direct memory access between plugins and their 
GUIs results in several problems. In particular, a 
large subset of many plugins’ functionality is inac¬ 
cessible except via the custom GUI. This severely 
limits the power of the system to intelligently au¬ 
tomate or otherwise control plugins. In addition, 
shared access to mutable data from multiple threads 
is an infamously error-prone situation, made even 
more difficult with the addition of real-time require¬ 
ments. It is all too common for a poorly written 
GUI to cause audio drop-outs, or crash the plugin 
entirely. Direct access to plugin internals is occa¬ 
sionally necessary (for visualisation in particular), 
but is an inherently flawed approach to plugin con¬ 
trol in general. 

The solution to this problem is a classic one: sep¬ 
arate the plugin and its user interface, and have the 
two communicate via messages. If these messages 
are meaningful (i.e. not opaque), the plugin can be 
controlled in the same way from any source: the 
GUI, the host, other plugins, scripts, and so on. Tra¬ 
ditionally, MIDI is used for this purpose, but MIDI 
has significant limitations. Ingen supports sending 
arbitrary messages between hosted plugins and their 
UIs (including MIDI), and is controlled entirely via 
messages itself. 

Control via standard and portable messages is the 
key to building audio components that can be de¬ 
ployed in many different scenarios. Designing the 
system fundamentally around this principle (rather 
than “bolting on” partial support for remote control) 
ensures that all interfaces to the system enjoy the 
same power. 

2.3 Polyphony 

Though inspired by modular synthesizers, Ingen 
does not seek to emulate the limitations of hard¬ 
ware. Polyphony in particular is an important fea¬ 
ture where software has a distinct advantage. This 
is an area where extreme minimalism is counter¬ 
productive: though it is possible to build a poly¬ 
phonic synth manually in a monophonic modular 
by replicating voices, this is a burden on the user. 
Instead, Ingen implements polyphony internally. 
Nodes can simply be flagged as polyphonic, and 


they will be replicated as necessary. Polyphony 1 is 
a property of the containing graph, i.e. if a graph 
has polyphony p, all nodes in that graph have either 
1 or p voices. Any connection between polyphonic 
ports is a polyphonic connection, and any connec¬ 
tion from a polyphonic port to a monophonic port 
mixes down all voices. 

2.4 Data Types 

Ingen supports many data types, including audio, 
“control voltage” (CV, audio-rate numeric controls), 
and events in any format such as MIDI. A port 
transmits either signals or sequences: audio and CV 
are the only signal types, everything else is a se¬ 
quence. Sequences are a series of “events” or “mes¬ 
sages” transmitted in-band with audio. LADSPA- 
style control ports are control-rate signals from the 
point of view of the plugin, but in Ingen are exposed 
as sequences of floating point numbers to allow con¬ 
trol changes to be transmitted with sample accuracy. 

The ability to work with many data types is pow¬ 
erful, but requires the user to understand the types of 
different ports. Ingen distinguishes port data types 
by colour, and also adds hint symbols as shown in 
Table 1 . Signal and sequence ports are distinguished 
by shape: signal ports have rounded borders (sug¬ 
gesting continuous), and sequence ports have bev¬ 
elled borders (suggesting discrete). A symbol is 
also composed on the type hint, for example, a real 
number signal (CV) is tagged ‘M’, and real number 
messages are tagged ‘M’. 


Symbol(s) 

Data type 


Audio (floating point) 

R 

Real (floating point) 

Z 

Integer 

M 

MIDI 

□,0 

Boolean 


Patch message 


Table 1: Type hint symbols for ports. 


Despite the many different data types, Ingen at¬ 
tempts to preserve the “anything to anywhere” abil¬ 
ity of classic modular synthesizers wherever possi¬ 
ble. For example, a float message output can be con¬ 
nected to a CV input; Ingen will automatically write 
the CV buffer as if the signal were continuous. 

2.5 Inter-Plugin Communication 

Many plugins must communicate both audio and 
messages, a typical example being MIDI synthe¬ 
sizers. Both are transmitted in Ingen in the same 
context to allow sample-accurate real-time message 

1 As opposed to the boolean polyphonic. 









handling, and avoid threading issues. This is dis¬ 
tinct from some systems, such as Pd, where mes¬ 
sage transmission follows different rules than signal 
transmission. In other words, messages in Ingen are 
in-band with audio signals. 

The benefit of this approach is a single consis¬ 
tent concept of real-time: plugins have one run() 
method which processes all inputs and emits all 
outputs synchronously. However, some plugins 
must perform non-real-time operations in response 
to messages. For example, a sampler plugin may 
need to load samples from disk. 

The LV2 worker extension solves this problem. 
The worker extension provides a simple API for 
plugins to schedule a callback to be called “soon” 
in a non-real-time thread, and a mechanism for re¬ 
plying back to the audio thread in a later cycle. This 
makes it possible for plugins to perform non real¬ 
time operations, but the API is designed such that 
its use is inherently real-time safe, and plugins do 
not need to use any non-portable threading libraries. 
Having this mechanism implemented by the host 
has performance benefits as well, for example, the 
host can share one ring buffer and worker thread for 
all plugins. This can dramatically reduce the mem¬ 
ory consumption when many plugins are loaded. 

3 Architecture 

3.1 Model 

Ingen uses a simple data model to describe all com¬ 
ponents of a graph. Each object has a unique path 
(like /fx/verbl) and a set of key:value properties. 
Keys are URIs (making state meaningful), and val¬ 
ues may have any type. Essentially, everything is a 
hierarchical tree of dictionaries. 

The use of a consistent data model allows for 
a very simple protocol to perform a large number 
of operations. Rather than adding “commands” to 
the interface for every new feature, changes arc im¬ 
plemented in terms of property changes. Only a 
few methods are required to allow arbitrary property 
changes, so this allows for a powerful yet stable pro¬ 
tocol. There are no issues with breaking the number 
or order of arguments, since properties have no or¬ 
der. New information can be added freely without 
requiring any changes to old code. 

3.2 Protocol 

The Ingen protocol itself is very similar. Messages 
are built from LV2 Atoms [Robillard, 2014], par¬ 
ticularly “Object” 2 which is a dictionary with URI 
keys and any type of value. 

2 This is an “object” in the JSON sense, not as in object- 
oriented programming. 


The LV2 Patch extension defines several mes¬ 
sages, similar to HTTP and DAV methods, which 
can be used to access and manipulate the graph. The 
simplest is a Get, which requests a description of 
the given subject: 

[ 

a patch:Get ; 

patch:subject </osc> ; 

] 

The response describes the subject in the same 
format, in this case the plugin instance, or block: 

</osc> 

a ingen:Block ; 

lv2:prototype <urn:someplugin> ; 
ingen:canvasX 42.0 ; 
ingen:canvasY 24.0 . 

Manipulation is similar. For example, a Put mes¬ 
sage can be used to create the above block: 

[ 

a patch:Put ; 

patch:subject </osc> ; 

patch:body [ 

a ingen:Block ; 

lv2:prototype <urn:someplugin> ; 
ingen:canvasX 42.0 ; 
ingen:canvasY 24.0 ; 

] 

] 

Syntactically, this says “I am a Put message, with 
subject/osc, and body [ a ingen: Block ...]”. 
The definition of patch:Put and the associated 
properties gives us the meaning: “put this block at 
/osc”. 

The short names here are abbreviations of 
URIs, for example, patch:Put expands to 
http://lv2plug.in/ns/ext/patch#Put. URIs 
are used here to provide a global namespace, but 
when properly documented, also provide trans¬ 
parency. For example, the above URI leads to 
documentation which describes the meaning of a 
patch:Put. This documentation is also machine 
readable to support intelligent tools. For example, a 
patch:Put must have one patch: subject prop¬ 
erty, and the same tools used for FV2 plugin vali¬ 
dation can ensure this restriction is obeyed. Note, 
however, that no Internet access is involved in han¬ 
dling messages; properly documenting URIs is sim¬ 
ply a best practice for convenience and tool support. 

There are similar messages to delete elements, set 
properties (including control values), and so on. All 
messages are defined in the FV2 Patch extension, 
which is also used by some plugins for control (for 
example, the FV2 example sampler uses this vocab¬ 
ulary to load samples). 



3.2.1 Serialisation 

Conceptually, Ingen uses the same protocol every¬ 
where. However, the above text serialization would 
only be used over a network, or shown for debug¬ 
ging purposes. When running in the same pro¬ 
cess, messages are instead serialised as binary LV2 
atoms for increased performance. These two en¬ 
codings are conceptually identical and differ only 
in representation. Similarly, plugins inside Ingen 
which communicate with atoms are connected di¬ 
rectly, with no serialisation. 

The same textual serialisation used in the remote 
protocol is used when saving graphs. Conceptually, 
the Ingen protocol can be considered a stream of 
patches to the saved graph (hence the name of the 
LV2 Patch extension). For example, the descrip¬ 
tion of the /osc block returned by the server in Sec¬ 
tion 3.2 could be a literal snippet of a saved graph 
file. Ports use the standard LV2 vocabulary, so In¬ 
gen graphs can be loaded by applications with LV2 
support, with no special Ingen support required. 

3.3 Event Handling 

Building and manipulating a graph of plugins re¬ 
quires operations that are not real-time safe. To al¬ 
low live editing without dropouts, Ingen must avoid 
all such operations (such as memory allocation or 
mutex locking) in the audio thread. 

Conveniently, message-based control lends itself 
to an event-oriented implementation, which makes 
for an elegant solution to this problem. All opera¬ 
tions in Ingen are implemented as events which are 
triggered by the receipt of some message. An event 
has three phases: 

1. Pre-Process: Upon receipt of the message, per¬ 
form any non-real-time operations necessary 
before the change can be applied (e.g. instan¬ 
tiate a plugin). When finished, push the event 
into a queue for the audio thread. 

2. Process: In the audio thread, apply the changes 
prepared in the pre-process stage (e.g. insert an 
instantiated plugin into a graph). After this, the 
change is effectively complete. When finished, 
push the event (including references to any re¬ 
sources that need to be freed) into a queue for 
post-processing. 

3. Post-Process: Clean up any necessary re¬ 
sources, and broadcast the change to all clients. 

4 Examples 

The most straightforward use of a modular is to 
build chains of effects plugins. Though simple, 


even this provides an improvement over what is eas¬ 
ily achievable in hosts with a strictly linear signal 
path. For example, processing the left and right 
channels separately in a DAW like Ardour can be 
achieved this way without complicating the ses¬ 
sion’s bus routing. 

More interesting is to custom-build instruments. 
Figure 1 shows an example of an extremely sim¬ 
ple polyphonic synthesizer, with only one envelope, 
saw oscillator, and low pass filter. 

Ingen allows graphs to be nested, and has no re¬ 
strictions on the type or number of ports present. 
For example, Figre 2 demonstrates adding sidechain 
compression to a synthesizer graph. 

It can be useful to combine existing high-level 
plugins with more low level components. For exam¬ 
ple, Figure 3 shows a graph which contains multiple 
instruments. A MIDI filter [Gareus, 2014] plugin is 
used to send automatic chords to an electric piano, 
while the input note is sent to a synthesizer. 

5 Future Directions 

Ingen is currently useful as an environment for host¬ 
ing plugins with flexible routing. Its architecture 
allows it to function in a diverse range of environ¬ 
ments, which has been the focus of development to 
date. 

One goal for future development is to become a 
more powerful programming environment. Since 
plugins are free to communicate with arbitrary mes¬ 
sages, the necessary infrastructure is already avail¬ 
able, but an appropriate set of plugins is missing. 
Existing systems like Max/MSP and Pd are very 
mature in this respect, but use a different model than 
Ingen and LV2. In particular, it will be interesting 
to investigate how to exploit meaningful messages 
to provide a powerful modular programming envi¬ 
ronment. 
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Figure 1: A simple polyphonic synthesizer. 
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Figure 2: Adding sidechain compression to a synthesizer. The “Synth” block shown here is a nested Ingen 
graph (which can be edited by double-clicking in the interface). 
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Figure 3: Simple MIDI processing to play the input root note on a DX10, and accompanying chords on an 
ePiano. Shown in vertical mode. 










































































































































































